Metrics for Learning Object Metadata

Previous research has set the foundation of Learning Object Technologies; unfortunately just the foundation is not enough to convince instructors and learners to use the technology. Mature tools are needed in order to breach the early-adopter mainstream gap. Following what has been done with other related technologies, this work presents research to create automated measurements (metrics) that could enable the creation of a new generation of smarter and friendlier learning object applications. The methodology for the proposal and test of the metrics is discussed, along with early encouraging results in the area of metadata quality metrics. Finally we present possible applications of the current and future results of this research. 1. Research Context and Justification During almost 15 years of research, the foundation of Learning Objects Technologies has been set. There are standards that define the metadata that describe a learning object [1] and how to sequence it [2]. Thanks to these standards, Learning Management Systems (LMS) are able to import and export learning objects of different granularity. There are several repositories worldwide where the instructors can publish the learning objects that they create and search for learning objects published by peers [3]. Thanks also to standardization [4], these repositories can query each other and present the user with a considerable amount of results. Despise all the development in this necessary foundation, the tools that the end user access to index, search, integrate and re-use learning objects are still immature if we compare them to similar tools used in related fields. For example, web pages creators or papers publishers do not need to manually index their work [5] [6]. The basic text-matching or field-matching techniques of current learning object search tools are not enough to sort the huge amount of results returned by federated queries, a problem that has been solved in the World Wide Web thanks to PageRank-like algorithms [7]. There is not readily available equivalent in the learning object community for the Amazon’s book recommending feature [8]. This lack of maturity is reflected in the low level of adoption of learning object technologies among instructors and learners [9] [10]. In order to improve the adoption of learning object technologies, smarter and friendlier end user tools must be developed. These tools should capitalize the vast amount of information that is present in the learning object metadata and other sources as context and usage. To be exploitable, that information should be automatically measured and processed to extract deep knowledge of the characteristics, relations, usefulness, behavior and recommended usage of individual learning objects, as well as, complete learning object repositories. This research will create and test automatic quantitative measurements (metrics) for learning object metadata. The objective of the metrics will be to find useful calculations that use intrinsic and extrinsic information to improve the performance and usability of the current tools. For example, if the number of times an object is reused (metric) is a good predictor of the relevance of a learning object (learning object characteristic), it could be used inside the sorting algorithm of federated search application (tool). We have called this initiative “Metrics for Learning Object Metadata”. The idea of using metrics to automate or improve tools or procedures has been borrowed from other fields: Software Engineering uses metrics to semi-automatically determine the cost and duration of software projects [11]. Scientometrics creates metrics to automatically predict the “impact” of a journal or gain insight about the research environment at a given moment and field [12]. Webometrics creates metrics to determine the relevance of a web page [13] (for example Google’s PageRank metric [7]). This new small research area, Metrics for Learning Object Metadata (Learnometrics), will create metrics based on the information present in the metadata record(s) of a learning object and the contexts where it is used to enable the creation of a new generation of Learning Object end user tools.